Stanford University ’ s Arabic - to - English Statistical Machine Translation System for the 2009 NIST MT Open Evaluation

This document describes Stanford University’s first entry into a NIST Arabic-English MT evaluation. We describe two main improvements over a previous Chinese-English submission (Galley et al., 2008): a hierarchical lexicalized reordering model (Galley and Manning, 2008) and a technique for performing minimum error rate training (Cer et al., 2008) that outperforms the standard Powell method. 1 System Description 1.1 Phrase-based translation system The core engine of our system is Phrasal, a phrasebased decoder similar to Moses (Koehn et al., 2007). In its baseline configuration and basic set of features, Phrasal replicates Moses almost exactly, and differs only in the way the decoder breaks ties between translation hypotheses that have the same score. Unless otherwise indicated, we use the same default parameters as Moses (e.g., same recombination heuristic, same maximum number of translation options for each input phrase). Phrasal uses a log-linear approach common to many state-of-the-art statistical machine translation (SMT) systems (Och and Ney, 2004). Given an input Arabic sentence f, which is to be translated into an English sentence e, the decoder searches for the most probable translation ê according to the following decision rule: ê = argmax e {P(e|f)} = argmax e { M ∑ m=1 λmhm(f,e)} where hm(f,e) are M arbitrary feature functions over sentence pairs, such as translation probabilities. Our system incorporates the following 17 feature functions: • Two phrase translation probabilities Pml(e| f ) and Pml( f |e), computed using the (unsmoothed) relative frequency estimate Pml(ē| f̄ ) = count(e, f )/ ( ∑ e′ count(e′, f ) ) , where f and e constitute a pair of aligned phrases. • Two lexical translation probabilities Plex(e| f ,a) and Plex( f |e,a), similar to those presented in (Koehn et al., 2003): Plex(e| f ,a) = n ∏ i=1 1 |{i|(i, j) ∈ a}| ∑ (i, j)∈a p(ei| f j), where n is the length of the phrase e, and a is the internal word alignment between e and f .1 • Eight hierarchical lexicalized phrase reordering scores for each phrase pair. We select from four types of orientations (monotone, 1Distinct instances of a given phrase pair (e, f ) may be observed with different internal alignments. In these cases, we select the most frequent alignment (like Moses but in contrast to (Koehn et al., 2003)). About 0.3% of our phrases have lexical translation probabilities that differ from Moses since our feature extraction implementation breaks ties between alignment counts differently. However, we observe no impact on MT performance. swap, left discontinuous, and right discontinuous) and model both left-to-right and rightto-left re-orderings. Laplace smoothing with λ = 0.5 is applied to the lexicalized re-ordering probabilities. More details about this model can be found in (Galley and Manning, 2008). • Two language models, from Gigaword and Google n-grams. • Word penalty as in (Koehn et al., 2007). • Phrase penalty as in (Koehn et al., 2007). • Linear reordering penalty as defined in (Koehn et al., 2007). The weights of these feature functions were set using an improved version of minimum error rate training (MERT) (Och, 2003). Specifically, we used a stochastic method and two regularization strategies that are described in (Cer et al., 2008), which shows that this approach is superior to both Powell’s method and the variant of coordinate descent found in the Moses MERT utility. Our system was tuned using MT06 (LDC2007E59). We did not tune different systems for different genres. The decoder used a distortion limit of 5. Stack size and n-best list sizes were set to 500 (Moses’s defaults are respectively 200 and 100, which we found less effective). After decoding, hypotheses are selected using the minimum Bayes risk criterion (Kumar and Byrne, 2004).